Add deterministic AI reporter qualification gates - #5999

Independent /pr-agentmerge review completed; fixes are in ac3fd75.

Findings fixed

• Full-log qualification previously inspected only the file reporter artifact. It now validates the AI-emitted absolute path, exact artifact correlation, format/completeness, existence, owner-only mode, required lifecycle content, external evidence, and failure/session/diagnostic correlation.
• Actionability now checks exact public context values and required privacy markers instead of key presence.
• The uniform 4 KiB synthetic blob was replaced by scenario-specific deterministic evidence. Aggregate ratios are supplemented by compact-case and per-case baseline gates so one oversized fixture cannot hide a regression.
• Local-sensitive producer identity is retained in the owner-only full log, while secret producer identity/values remain redacted. Secret diagnostics preserve honest aggregate counts and truncation without exposing code/category/detail or suppressing fallback errors.
• Future eligibility now requires both a passing qualification result and an explicit accepted privacy prerequisite. COPILOT_CLI/configured detection alone stays ineligible; RUSH_REPORTER=legacy remains authoritative. No telemetry implementation or Rush 6/default selection behavior was added.
• Added an adversarial reporter regression test: dropping artifactAvailable drives full-log to 0% with all affected case names in the CI diagnostic.

Corpus evidence

• 12 cases: 10 failures plus success and warning-only controls covering bootstrap/environment, configuration, input, dependency tool, operation/build, cache, network/auth, plugin, cancellation, internal, and success paths.
• Three independent standalone corpus executions were byte-identical across every normalized SHA-256.
• Gates: actionability/privacy/full-log/stdout/warnings/determinism all 100%; max AI case 976 B; compact max 969/2048 B; worst comparable per-case 68.47% of legacy and 72.00% of plaintext; aggregate 43.77% and 46.51%.
• Machine results were scanned for temp paths, fixture secrets, secret producer identity, and private plugin identity; none were present.

Validation

• Reporter build/lint/API Extractor and 319 tests passed.
• rush test --to @microsoft/rush passed the reporter, rush-lib, apps/rush, and dependency chain.
• rush check and stacked rush change --verify --target-branch origin/copilot/reporter-r5b-demo-reporters --no-fetch passed.

State / blockers

• PR remains based on Add direct Rush reporter demo path #5997, mergeable/clean, and auto-merge is disabled.
• No review threads existed to resolve.
• GitHub currently reports only the CLA check and no Actions run for this stacked head.
• Harden reporter telemetry privacy projection #5990 and Add direct Rush reporter demo path #5997 remain open; [rush reporter][R8] Qualify AI reporter auto-selection #5981 must remain open until both prerequisites merge, the qualification passes on main, and any approved activation remains gated by both prerequisites.

Combined deep review of current head ac3fd75530.

1. HIGH local-sensitive messageEmitted text can be copied verbatim into AI stdout fallback output. Only public text may enter AI output. Use a redacted fallback and protected log reference otherwise.
2. LOW the oversized-record marker relabels non-public envelopes as public while retaining source and scope. Preserve privacy or remove those fields.

Fixed both combined review findings in c46a4ea8036805e70de1143f25d206b4c653e42c.

• AI fallback text is emitted only for public envelopes. Local-sensitive and secret fallback errors are count-only, marked truncated, and retain the protected full-log reference.
• Non-public JSON oversized markers keep their original privacy and remove source and scope. Public markers retain scope and child-session correlation.
• The corpus now covers mixed-privacy fallback messages, oversized local-sensitive payload/source/scope sentinels, secret producer identities, safe machine output/results, and useful owner-only logs.

The three R8 commits are rebased directly onto 97f807ed6549387b6aab2eccec197123196b725d. All 13 cases passed across three identical runs. Reporter, rush-lib, and apps/rush tests passed, along with API/lint, Rush check, and stacked change verification.

Head is pending review. Auto-merge remains disabled.

Four-review gate on c46a4ea803 confirms the AI fallback and oversized-marker fixes. One HIGH privacy issue remains: normal-sized secret JSON events still retain scope and parentOperationId, which can expose private project, phase, or operation names on machine stdout.

Required fix: use an allowlisted secret-envelope projection that removes contextual metadata not guaranteed opaque, while preserving protocol and ordering fields. Extend the corpus with sentinel scope values.

Dismissible low follow-on: decide whether local file:// and json:// sidecars should retain local-sensitive message text or match stdout redaction.

Fixed the HIGH secret scope metadata finding in 6dac2ffdcdb761232829e9160ec13031d3fb65a6.

• Shared redaction now projects secret envelopes through a strict allowlist. Parent, command, operation, project, phase, scope, and producer component metadata are removed.
• JSON normal and originally oversized secret messages and diagnostics use the same redacted contract.
• AI ignores secret command and operation lifecycle context while preserving safe diagnostic, fallback count, and command completion behavior.
• Corpus probes now use sentinel command, operation, project, phase, parent session, parent operation, source, message, and diagnostic values. None reach machine stdout or qualification results.

Reporter build, lint, API Extractor, and 336 tests passed. All 13 corpus cases passed across three identical runs. Reporter, rush-lib, apps/rush, Rush check, and stacked change verification passed.

Local-sensitive and full-log behavior are unchanged. The low file-sidecar policy follow-on remains separate. Head is pending review and auto-merge remains disabled.

Final combined review of 6dac2ffdcd found no critical, high, or medium actionable issue. Secret JSON and AI records now remove scope, parent, command, operation, project, phase, and producer component metadata while preserving safe protocol and count fields. Qualification gates remain deterministic and automatic selection stays disabled.

Dismissible low follow-ons:

• Decide whether owner-only file grouping should suppress secret operation headings and raw secret output beyond the documented raw-child limitation.
• Decide whether local file sidecars retain local-sensitive message text or match stdout redaction.

This head is low-only.

Deep review of current head ec9238b found no critical, high, or medium actionable issues. Local-sensitive producer identity is preserved, secret envelopes remain allowlisted, and the qualification suite passes. Merge remains gated by #5997, #5990, and maintainer approval.